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A study was made to deteraine the reliability of 
subjective adult judgeaent in the selection of filas for priaary 
educable mentally handicapped (EMH) children. A selection of five 
filas, all prerated according to standardized technique.^ and 
designated appropriate for primary level students, were presented for 
vietfing by a priaary class of heterogeneously grouped EMH students. 
It was expected that the wide range of acadeaic abilities and social 
maturity would be reflected in the type of response that the class as 
a whole would make to the filas. A situation was therefore created 
whereby attempts were made to subjectively predict the type of film 
to which the class would make the most positive response. The quality 
of response was measured in teras of attending behavior, length and 
content of the fila, and the number of correct answers to the 
accoapanying question set. Observation data revealed that priaary EMH 
students benefitted from expiosars to all five filas. It was concluded 
that subjective judgement regarding the educational value of films 
for heterogeneously grouped EMH students at the primary level tends 
not to be reliable. (Author) 
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SELECTING FILMS FOR PRIMABY EMH CHILDREN 
ABSTRACT 

A study to determine the reliability of subjective adult judgment 
in the selection of films for priinary EMH students. A selection of 
five films, all pre-rated according to standardized techniques and 
designated appropriate for primary level students, were presented for 
viewing by a primary class of heterogeneously grouped EMH students. 
It was expected that the wide range in academic abilities and social 
''^'^^^ ''''m^^^ class as 

a whole would make to the films. A situation was therefore created 
Whereby attempts were made to subjectively predict the type of film to 
which the class would make the most positive response. The quality of 
response was measured in terras of attending behavior, length and content' 
of the film and number of correct answers to the accompanying question 
set. Observation data revealed that primary EMH students benefitted from 
exposure to all five films. It was concluded that subjective Judgment 
i-egarding the educational value of films for heterogeneously grouped EMH 
students at the primary level tends not to be reliable. 
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SPECIAL REPORT No. 743 
COMPUTER-BASED PROJECT for the EVALUATION 
of MEDIA for the HANDICAPPED 

T i t I 6 1 SELECTING FlUiS FOR PRIMARY EWH CHILDREN 

BACKGROUND 

The Computer Based Project for tlie Evaluation of Media for the Handicapped, 
based on contract ftORG-9-423617-4357 (616) between the Syracuse (N.Y.) City School 
District and the Media Services and Captioned Films Branch, Bureau of Education 
for the Handicapped {United States Office of Education) for the five year period 
July 1,_ 1969 through June 30, 1974. The major goal is to improve the instruction 
of handicapped children through the development and use of an evaluation syatcm 
to measure the instructional effectivener^s of films and other materials with 
educable mentally handicapped (EMH) children, in-service training and media suppor' 
for special teachers, and studies related to the evaluation process and the 
populations used. 



The Project has concentrated on the 600 films and 200 filmstrips from the 
Media Services and Captioned Films (BEH - USOE) depository! however, specific 
packages from Project LIFE, various elementary math curricula, and selected 
programs from Children 's TV Workshop have also been evaluated. The evaluation 
model used requires that i 1) objectives of materials be specified and written* 

2) instruments be constructed to test and measure ef fecuivenessj and» 3) children 
be the major sources of evaluation information. A number of instruments and 
methodologies are employed in the gathering of cognitive and affective data from 
900 EMH children and 80 special teachers to make the effectiveness decisions. 
Over half of the EMH population can neither read or write; therefore, a unique 
Student Response System (SRS) is employed, consisting of a twenty station G.E.- 

• 1000 SRS which can be operated in a group or individual recording mode and is 
connected to a remote computer system . The computer capabilities consist of 
remote telephone connections to the Rcme (H.Y. ) Air Development Command, the 
Honeywell time-shared network, and the Schenectady (N.Y.) G E Research and 
Development Center! and batch mode capabilities of the Syracuse City Schools, 
Syracuse university, and various commercial sources. 

In-service and media support activities provide on-the-job training for 
teachers, teacher aides, equii^ment, and materials to the special teachers in 
the city schools. The research activities have centered around investigations 
and special problems related to the development of the evaluation ihodel. The 
four major areas considered aret 1) testing effects, 2) captioning effects, 

3) special student characteristics; and, 4) evaluation procedures validation. 

Documentatior. of the major activities appear in the five annual reports 
and the 600 evaluations prepared on materials used. Staff members wore encouraged 
to prepare special reports and the attached paper is one of these. The opinions 
expressed in tliis publication do not necessarily reflect the position or policy 
of the Computer Based Project, the united States Office of Education, or the 
Syir^puse City School District, and no official endorsement by any of the agencies 
sliottld be inferred. 
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SELECTING FILMS FOR PRIMARY EMH CHILDREN 

During the past five years i the Computer Based Project has been 
involved with the development of an evaluation model based on student 
responses to instructional materials, and the study of the interaction 
between student characteristics and media characteristics. Data anal- 
yses of student and media characteristics and the relationship of stu- 
dent responses to media have been the basis of a number of reports. 
The subject of this paper is the selection of films for primary educable 
mentally handicapped (EMH) children, Observations reported are based 
upon weekly contact with students from one classroom from January to 
March who had been formally trained to use the student response system _ 
(SRS) during the month of October ♦ In working with, younger and wore 
heterogeneously grouped students it was observed that a more deliberate 
effort may be needed in selecting films for their viewing; 

The authors of two CBP pilot investigations, (Salon » 197i*i Grayson, 
1972) have indicated that variables such as length of presentation, 
color, characteristics of captions, etc, have effects on the attending 
behavior of viewers , and therefore probably on their cognitive behavior 
as well. Salon concluded that'"*/, by adapting the materials and items 
used to the capabilities of the subjects, (i.e. , by dealing only with 
simple and concrete information), primary EMH children can benefit from 
the content of primary level films with a minimum of assistance." ^ 



Recommendations speak to the need for careful consideration of the 
appropriateness of the film^s question format and items, Grayson 
concluded that ''CBP findings on attention seem to indicate that dif- 
ferent presentations have different profiles; that younger (6-10 
years) EMH children have a greater range of variance in attention than 
older children; that a relationship exists between high mean attention 
and content learned; and that media can be grouped and ranked on atten- 
tion variables, 'V The evidence provided by these studies supports the 
contention that certain selective criteria must be considered in 
assigning films for primary EMH students in order to ensure their 
optimal performance, i.e* I measured by attending behavior and cognitive 
learning, The essential question raised in this study isj To what 
extenti if at allj subjective adult Judgment is reliable in the selection 
of films for primary EMH students, 

METHOD 

Subjects , From Januarys-March > a total of five sessions , an attempt 
was made to estimate the response of one class of students to films 
selected for their viewing. The assigned films had been rated by 
project coordinators and considered appropriate for primary students • 
In light of the heterogeneity of this group I viewed in terms of the 
students' general academic characteristics ^ i*# e,i level of academic 
maturity and intellectual functioning. It seemed apparent that some 
films would be more appropriate than others. The range of character- 
;-lstics and abilities of students in the class are listed in the follow- 
ing tables 



TABLE I Range of Student Characteristics 





Lowest 


Highest 


Chronological Age 


5h yrs • 


9 yrs. 


Mental Age 


^ yre* 


8 yrs. 




59 / 


101 


Reading Score 


.5 


1,2 



Film Se lection 

In a five-week period two filjns, (A and B ), scheduled for viewing 
by this class were considered by the investigator to be too abstract 
and above the level of appropriateness for younger primary students. 
It was anticipated that the students' response to the question sets 
would be poor; Based upon this subjective speculation another two 
films (C and D), Judged to be more appriate, were substituted, At a 
later date films A and B were shown to the class and data collected from 
the assigned question sets. The major criterion used to determine the 
educational appropriateness of a fiL was the number of correct re- 
sponses made by the student to the question set after viewing the film* 

The following paragraphs deal with a weekly account of the investi- 
gator's experiences in attempting to pre-determine the relative educa- 
tional value of specific films scheduled for presentation to this class. 



REPORT ON WEEKLY FILM SELECTION 
Media Number Order of Presentation 

Film C #1001*, '^Country Mou^e* and City Mouse'* 1st week 

Film D #1032, ^'Summer on tHe Farm** 2nd week 

Film A #2379i "Eating in a Restaurant" 3rd week 

Film E #2m2, ''The Shoemaker and the Elves" ^th week 

Film B #2150, "Johnny Appleseed" 5th week 

O\)servations 

January 28, 197^, filni A, originally scheduled for viewing, was 
replaced by film C. This substitution was made after film A had been 
previewed and considered of probable high interest appeal , but too 
lengthy, (15 minutes) to hold the attention of primary EMH students, 
A further deterrent was the rather long gaps in the monoiogue to allow 
deaf children the time and opportunity to read lips , It was felt that 
these periods of delay would be too distracting for young EMH students 
who could not read the captions on the picture, The students seemed to 
enjoy film C and the attending behavior was relatively high; the mean 
percent of primary students watching this film was approximately 82, 
The teacher indicated on a checklist that the vocabulary was above the 
level of the class, but that the concept and question levels were 
about right; Seven questions were presented to the class, Of ten pu- 
pils, five answered from 60*10055 of the items correctly, The remaining 
five students answered 505^ ov lets of the items correctly • 

February 6, 197U, as was the case the previous week, the filiQ 
originally scheduled for viewing was substituted in favor of a shorter 



film; one with a more "simple" story line and vocabulary, In view of 
the general level of functioning of this class, film B was considered 
to be too lengthy {ih minutes) to hold the attention of the majority of 
students, Moreover, It was felt that vocabulary and concepts were 
relatively abstract, i.e., pioneer, settlement, transplant, seedling, 
etc I The following comment about the film is noted in the Berkeley 
Edition of the Lesson Guide, p, 29 "..^ it is basically an intermediate 
social studies film, but could be used to enrich studies of language 
arts and American literature," 

Film D was presented to the class as a substitute for film B. The 
mean percentage of primary students watching this film was approximately 
85. The teacher feedback indicated that the vocabulary was above the 
level of appropriateness, but that the concept level was about right, 
Response to the question set containing both pre and posttest questions 
was relatively good, Four pupils answered of the twelve questions 

correctly. Five pupils answered 505{ or less of the questions correctly, 

February 13, 197^ • in view of the overall response to fllrn A and 
its corresponding question set there would appear to, have been no reason 
not to show this film as previously scheduled. The attending behavior 
was good, the mean percentage of primary students watching being about 
8U, The teacher felt that the vocabulary, concepts and questions were 
appropriate. Data analysis revealed that these primjary students did 
relatively well on the question set. For example, three students an- 
swered 60-70J{ of the questions correctly, Two students answered 50% o^^ 
the questions correctly while the remaining five students answered lee^ m 
than 30% of the questions correctly, 



February 27 > 197^ i film E, the assigned film was viewed as scheduled, 
The mean percentage of primary students watching this film was approxim- 
ately 85* Tlie length of the film (13 minutes) appeared not to deter the 
level of interest, although toward the end the children began to squirm 
in their seats. Data analysis revealed that three students answered 
80-9055 of the items correctly. Three students answered 50?{ of the items 
correctly, and four pupils answered less than 30% of the items correctly* 

March 13, 197^i film B was not shown when originally scheduled be- 
cause it was anticipated that these primary students would not be able 
to follow the plot or respond well to the question sett Data analysis, 
however, revealed that primary students responded as well as intermediate 
students to the question set, and that the mean percentage of primary 
students watching this film was 85 as compared ^to a mean percentage of 
95 for intermediate students. The primary classroom teacher felt that 
the vocabulary, concepts and questions were above the level of appropri- 
ateness for the class • 

The general response of students in this primary class to the question 
set was not as good when compared to previous films, i»e», film A or film 
D. Three students answered 80^ of the questions correctly. On t student 
answered 50^ of the questions and six students answered less than 50% 
of the questions correctly • The number of correct responses to this as 
compared to other question sets would appear to reflect and to support 
the fact that the range of abilities within the class is very broad (see 
Table 



RESULTS 



The following table depicts the relative response of students to 
the question sets of all five films. This data indicates that the 
level of performance regarding these films was approximately the same, 
and that the level of response among the ten students vis a vis each 
film remained relatively constant. 



10 
9 
8 
7 
6 

5 
k 



nook 



TABLE 2 

Number of Students Recording 
At Least Correct Response, 



#2150 








' #2379 ' 




#1032 • - ' ■■ 





2 3 14 5 6 7 8 9 10 
Number of items per question set 



11 12 



Ot the five films discussed in this report > serious question was 
raised about two, films A and B, Film A was initially rejected because 
it was considered too long and its delayed monologue a source of dis-^ 
traction for primary level EMH studer|tSt Film B was initially rejected 
because it was considered that the biographical sketch was too long and 
abstract in terms of the general story content, concept and vocabulary 
levels* Both films were, however, shown to the class at a later date 
to determine what type of response the students would actually make. 
The findings revealed that these students made as positive a 'response 
to films A and B as they did to films C, D and E, In fact student 
response to the question sets of films A and D was slightly better than 
for the other three films. 

. ■ . --^ ......... ......... SUMMARY... . 

Special attention has been given to the kinds of films presented 
to a class of primary EMH students. Attention was particularly paid to 
the films' content, length and to the question set| and an attempt made 
to subjectively predict the type of films to which the students would 
make the most positive response, The assumption was that films of a 
particular length and abstractness of content and dialogue would be too 
difficult for this particular group of primary EMH students to benefit 
from, i.e. , measured in terms of attending behavior and the number of 
correct responses to the question set. Contrary to expectation, there 
is no question but that these students benefitted from exposure to all 
five films. The author therefore concludes that it is extremely dif«? :^ ^ 
ficult to subjectively pre-determine the educational value of media for 
heterogeneously grouped EMH students on the primary level. 
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RECOMMENDATIONS 



In selecting films to be viewed by primary EMH classes whose 
populations are mixed, it is recommended that: 

1, The composition of the class, i,e»» range of abilities 

and performance levels of students, be taken into consideration • 

2, Consideration be given to the format, concept and vocabulary 
levels of the films and question sets, 

3/ Teachers not attempt to select films based solely upon 
their own subjective judgments. 

Teachers be consciously aware that adults tend to under- 
estimate rather than overestimate the learning potential and 
interest level of children of all ages. 
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